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COMPARISON OP THE VARIOUS METHODS FOR THE DIRECT CALCULATION OF THE 
TRANSMISSION FUNCTIONS OF THE 15-MICRON CO, BAND WITH EXPERIMENTAL 

DATA 


Academy of Sciences, U.S.S.R. 


I. INTRODUCTION /I* 

At the present time, the direct method of calculating the trans- 
mission function [1], based on numerical frequency integration of mono- 
chromatic transmittance, and Improved parameters of the thin structure 
of the absorption bands - which are used as the initial data - is widely 
used in solving different Inverse problems of satellite meteorology. 
Several publications describe different algorithms for the direct cal- 
culation with differing degrees of reliability [1-^]. There are several 
indetermlnancles in selecting the initial information: position, inten- 
sity, half-width, and particularly the forms of the absorption line con- 
tours and thilr dependence on physical parameters, different methods of 
calculating the Folgt function and numerical frequency Integration. The 
problem thus arises of comparing the calculation algorithms, both with 
reliable experimental data and with each other, in order to develop con- 
sistent, accurate, and economical programs for calculating the trans- 
mission functions. 

In this connection, steps were taken to comrare the different cal- 
culation methods with the results of recent laboratory measurements of 
the 15-Micron transmission functions for the CO2 band by M. 0. Bulanina 
et al [15]* The selection of this band for this comparison was based 
on very reliable experimental data given in numerical form, and obtained 
in a wide range of pressures, temperatures, and lengths of the optical 
beams. /2 

The results are given below of this comparison for four indepen- 
dent progi'ams for calculating the transmission functions: 

1) Program compiled by V. Deler [17]. 

1 
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2) Program compiled by V. M. Osipov [l 8 ]. 

3) Program developed at the Leningrad State University. 

k) Programs developed at the Moscow Physical-Technical Institute 

[ 10 ]. 

Below, these programs will be designated as PI, P2, P3, and pi< . 

We were faced with thj following problems In performing this 

study : 

1. To check the quality of different direct methods of calcu- 
lating the transmission functions by comparing the calculations with 
the experimental data. 

2. To compare the calculation methods with each other, and also 

to analyze the possible reasons for the lack of agreement In the results. 


2. Initial Spect r oscopic Information 

At the present time, the following parameters are generally used 
[ 11 ] to characterize the absorption cf the separate i-th spectral line; 

- position of the line center Ooi 

- Intensity of the line (for a reference temperature of T^) 

- half-width of the line ^usually for a unit pressure and 


of the lines fl. , it Is ne.oessary to define the ex- 

nc 

. Finally, for calcula- 


reference temperature T^) 

- Forms of the line contour f(v -'' 4 )* 

In order to calculate the dependence of the line Intensities on /3 
the temperature T, Information Is necessary about the energv of the 
lower level El In the form of a pre-exponential factor. For calculations 
of the half-widths 

ponent In the relationship •ii (f) 

tions of the emission transformation in various gas mixtures. It is 
necessary to take Into account the effectiveness of collisions of dif- 
ferent molecules by met.ns of the expansion effectiveness factors Be 
(e - index of expanding gas). 

In the general case, all of the parameters given above are dif- 
ferent for different spectral gases and absorption lines. Since fre- 
quently the information about these parmaeters Is far from adequate, 
different assumptions were used In the calculation methods used (PI, 

P2, P3, and P^4). 

Table 1 gives lnfor;.:ation about the sources of spectroscopic 
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Information' used In these programs, the position of the line centers, 
their intensities, and energies of the lower state, and half-widths of 
the lines . 


Table 1: SPliCTROSCOPIC INFORMATION USED IN PI. P2. P?. and pi<. 


Program (method) 

PI 

P2 

P3 

pi< 

Information about line 
centers. Intensities, 
and lower-state energy. 

Relations 
from [13] 

Calculated 
according 
to methods 
In [9, 12] 

Relations 
from [11] 

Relations 
from [13] 

Half widths of lines 

Constant 
. 07 

cm”^atm“^ 

Variable 
calculated 
from empi- 
rical for- 
mula In 
[1^] 

Variable 
relation 
from [11] 

Constant 
a »0.06^ 

cm-^atm”'*’ 

Exponent n 

0.5 

0.1 

0.5 

0.5 




3. Brief Description of Programs 

Following are the basic parameters Influencing the accuracy and 
rate of calculations: 

a) number of spectral absorption linos considered; 

b) form of spectral line used and method of calculating absorp- 
tion coefficient; 

c) method of numerical frequency Integration!; 

d) degree of utilization of data from preliminary calculations, 
different approximations, etc. 

The programs PI, P3, and P^ used the Foigt contour for the absorp- 
tion coefficient 


Ki(u;)= Hi 



where ( u) )ls the monochromatic coefficient of the 1-th spectral 
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line, and Lorentz and Doppler half-widths;}] ( CL,^0 ) Folgt 

function. The P2 program uses the Lorentz expression for the absorption 
coefficient . 

A large number of different methods are presently used for cal- 
culating the Folgt function. PI and P4 use the analytical expression 
obtained by V.S. Matveyev [ 15 ], and P3 uses the piece-wise approxima- 
tion using different expressions in different parts of the region in 
which the parameters CL and iO change. Ac compared with the methods 
in [ 15 ], this greatly accelerates th<> Cdlculation process for many of 
the values of CL and u) . It is well-known that if the expression for 
the frequencies close to the center of the Lorentz line (or the Folgt 
line) approximates quite accurately the real absorption contour, then /5 
its accuracy is poor in the wings of the lines. In this connection, 
these programs use different empirical methods for calculating absorp- 
tion in the wings of the lines; P2 and P^ use a modified Benedict con- 
tour [ 16 ], and PI and F3 use K * 0 at ^ ^ t where d is an 

empirical parameter (d * 10 cm~^) and d = 15 cm~^ for PI and P3, respec- 
tively) . 

In programs PI a :d P^, frequency integration is accomplished with 
a constant step fj , whose value depends on the pressure (for example, 
the integration step at p * 1 atm. is O.O 5 cm in PI, and 0.02 cm in 
P4 . Integration is performed in P2 by the ^■’mpson method with automa- 

_ o 

tic selection of the step and a relative accuracy of 10 . Integration 

is performed in P3 over the intervals between the pairs of adjacent 
spectral lines. Sections of a rapid change in the monochromatic trans- 
mission can be distinguished in these Intervals, and integration is 
performed by different methods in these sections and in the remaining 
sections of the spectral range. It must be noted that such an 
approach requires additional calculations for analyzing the behavior 
of the Integrand. 

To decrease the number of calculations, the total contribution 
of the far lines is first calculated in P^ for each spectral band, and 
the weak lines located in this band are calculated for P3. Using a 
constant Integration step, the authors of P^ tabulated the Folgt func- 
tion for given values of p T, and , greatly reducing the number of 
calculations . 

Table 2 summarizes the basic data characterizing the characteris- 
tics of these programs. The amount of initial information used is given. 




Table 2: BASIC CHARACTERISTICS OF THE PROGRAMS FOR CALCULATING PI 
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as well as a comparative estimate of the computational time. 

/7 

4, Calculation Conditions 

A comparison of the calculation results using the programs de~ 
scribed above was carried out for l4 specially-selected experimental 
conditions [5], whose characteristics are given in Table 3. The nu- 
merical values of the measured transmission functions for these 14 con- 
ditions are given .in Appendix 1. The spectral behavior of the half- 
widths of the equipment spread function a/cm";^ is described by the 
following formulas: 

= 0.01074 i) - 580^^0 ^ 634 

a. = 0-008041 i) - 2.39; 640 ^ 0 ^ 770 cm"^ 

Table 4 gives the mean square errors for different measurement 
conditions and different spectral ranges for the characteristics of 
the transmittance measurement accuracy. The errors in the spectral 
tie-in of the data do not exceed 1 cm“^. 


5. Results of Comparing Experimental Data and Calculation Results 

The following numerical characteristics were obtained for ana- 
lyzing the agreement between the calculated and experimental trans- 


mission spectra: 






where J is the number of the comparison conditions (Table 3), # is 
the frequency number M ^ number%of comjparison conditions used in 

the comparison; /f and / " experimental and cal- 

• g' ^ 1 JP * 

culated transmlssaon runctlons * respectively . The values of ^ U f 


/8 


and (S / J 
6 / Oi / 

tlonshlp 


/ are given In Table 5, and ^ / anc' 

are given In Figure 3. Figure ^ Illustrates the rela- 




where K ■ PI, P2, P^ , which provide a determination of the different 
calculation methods. P3 was selected as the reference program. An 
analysis of Table 5 and Figures 1-4 points to the following basic con 
elusions : 


1. The agreement between the calculated trarismlsslon functions 
obtained by means of the PI and P3 algorithms In terms of the average 

^ / J / Is almost Identical ( ^ ( J .) ^ 2%). The algo- 
rithms of P2 and P4 give much better agreement In terms of the average 

A / J /. 

2. In the majority of cases, the dl /ergence between 

PI and P3, and also between P2 and F4 In certain spectral regions, 
may be explained by a difference in the initial Information. Thus, 
for example, tn the region 645-650 cm”^ , the transmission values ob- 
tained by using P4 for different experimental conditions t?xceed the 
same values obtained using the P2 program by 2-6t, which is apparently 
explained by the fact that the transition / OiiOi-OuCOI / for 
the qI3qIo Isotope with a central frequency of 648.48 cm~^ was not 

taken into account In P4. 

3. Program P4 makes It possible to obtain the best agreement 
with experiment, on the average. 

4. The characteristic break in the graphs In the 667 and 68l /9 

cm~^ regions points to an error In the frequency tie-in of the experi- 
ment. The simplest Interpolation of the experimental data 


« ^ 

Improves the characteristics ^ / J / and (^ / J /by a factor 
of 1.5 - 2, and decreases the breaks In the graphs (see Figures 3 and 
4). Table 6 gives the characteristics ^ J ^ and ^ ! J / 
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after spectral correction of the data. 

5. Tnus, Is the most accurate and effective. A computational 
program for the transmission function and for atmospheric nonun Iformlty 
was used and successfully applied on the basis of P^. This program 
makes It possible to carry out calculations for wide spectral regions 
on an average computer (BESM-4, 28,000 operations per second) In about 
one hour. The comparatively rapid operation of Is achieved by ta- 
bulation of the line contour with an accuracy which Is sufficient for 
obtaining results about the transmission with an accuracv of about lilt. 

It Is planned to carry out similar comparisons for another Im- 
portant CO 2 band centered at 4.j microns. The experimental transmission 
functions [5] and the experimental conditions are given In Appendices 
3 and 2. 
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Table 3: Calculation Conditions 
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Table 5: Characteristics of ^ and ^ 
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Appendix 1: Transmission Functions for 15-mlcrons of the CO^ band. 
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Appendix 2: Experimental Transmission Functions (U.3 micron band) 

Region: 2200-2^25 cm~\ step - 1.0 cm"^ 
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Appendix 3: Experimental Transmission Functions of the Absorption Band at ^4.3 Microns. 

Initial Frequency - 2200 cm“^ , step - 1 cm~^ 
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